Question: Can an encoder-decoder architecture pretrained on a large dataset of longitudinal electronic health records improves patient outcome predictions? Findings: In this prognostic study of 6.8 million patients, our denoising sequence-to-sequence prediction model of multiple outcomes outperformed state-of-the-art models scuh pretrained BERT on a broad range of patient outcomes, including intentional self-harm and pancreatic cancer. Meaning: Deep bidirectional and autoregressive representation improves patient outcome prediction.

Importance: Social determinants of health (SDOH) are known to be associated with increased risk of suicidal behaviors, but few studies utilized SDOH from unstructured electronic health record (EHR) notes. Objective: To investigate associations between suicide and recent SDOH, identified using structured and unstructured data. Design: Nested case-control study. Setting: EHR data from the US Veterans Health Administration (VHA). Participants: 6,122,785 Veterans who received care in the US VHA between October 1, 2010, and September 30, 2015. Exposures: Occurrence of SDOH over a maximum span of two years compared with no occurrence of SDOH. Main Outcomes and Measures: Cases of suicide deaths were matched with 4 controls on birth year, cohort entry date, sex, and duration of follow-up. We developed an NLP system to extract SDOH from unstructured notes. Structured data, NLP on unstructured data, and combining them yielded seven, eight and nine SDOH respectively. Adjusted odds ratios (aORs) and 95% confidence intervals (CIs) were estimated using conditional logistic regression. Results: In our cohort, 8,821 Veterans committed suicide during 23,725,382 person-years of follow-up (incidence rate 37.18 /100,000 person-years). Our cohort was mostly male (92.23%) and white (76.99%). Across the six common SDOH as covariates, NLP-extracted SDOH, on average, covered 84.38% of all SDOH occurrences. All SDOH, measured by structured data and NLP, were significantly associated with increased risk of suicide. The SDOH with the largest effects was legal problems (aOR=2.67, 95% CI=2.46-2.89), followed by violence (aOR=2.26, 95% CI=2.11-2.43). NLP-extracted and structured SDOH were also associated with suicide. Conclusions and Relevance: NLP-extracted SDOH were always significantly associated with increased risk of suicide among Veterans, suggesting the potential of NLP in public health studies.

Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with an average of 3,000+ tokens. This task is challenging due to the high-dimensional space of multi-label assignment (155,000+ ICD code candidates) and the long-tail challenge - Many ICD codes are infrequently assigned yet infrequent ICD codes are important clinically. This study addresses the long-tail challenge by transforming this multi-label classification task into an autoregressive generation task. Specifically, we first introduce a novel pretraining objective to generate free text diagnoses and procedure using the SOAP structure, the medical logic physicians use for note documentation. Second, instead of directly predicting the high dimensional space of ICD codes, our model generates the lower dimension of text descriptions, which then infer ICD codes. Third, we designed a novel prompt template for multi-label classification. We evaluate our Generation with Prompt model with the benchmark of all code assignment (MIMIC-III-full) and few shot ICD code assignment evaluation benchmark (MIMIC-III-few). Experiments on MIMIC-III-few show that our model performs with a marco F1 30.2, which substantially outperforms the previous MIMIC-III-full SOTA model (marco F1 4.3) and the model specifically designed for few/zero shot setting (marco F1 18.7). Finally, we design a novel ensemble learner, a cross attention reranker with prompts, to integrate previous SOTA and our best few-shot coding predictions. Experiments on MIMIC-III-full show that our ensemble learner substantially improves both macro and micro F1, from 10.4 to 14.6 and from 58.2 to 59.1, respectively.

Pretrained language models (PLMs) have motivated research on what kinds of knowledge these models learn. Fill-in-the-blanks problem (e.g., cloze tests) is a natural approach for gauging such knowledge. BioLAMA generates prompts for biomedical factual knowledge triples and uses the Top-k accuracy metric to evaluate different PLMs' knowledge. However, existing research has shown that such prompt-based knowledge probing methods can only probe a lower bound of knowledge. Many factors like prompt-based probing biases make the LAMA benchmark unreliable and unstable. This problem is more prominent in BioLAMA. The severe long-tailed distribution in vocabulary and large-N-M relation make the performance gap between LAMA and BioLAMA remain notable. To address these, we introduce context variance into the prompt generation and propose a new rank-change-based evaluation metric. Different from the previous known-unknown evaluation criteria, we propose the concept of "Misunderstand" in LAMA for the first time. Through experiments on 12 PLMs, our context variance prompts and Understand-Confuse-Misunderstand (UCM) metric makes BioLAMA more friendly to large-N-M relations and rare relations. We also conducted a set of control experiments to disentangle "understand" from just "read and copy".

深度神经网络中的建筑进步导致了跨越一系列计算机视觉任务的巨大飞跃。神经建筑搜索（NAS）并没有依靠人类的专业知识，而是成为自动化建筑设计的有前途的途径。尽管图像分类的最新成就提出了机会，但NAS的承诺尚未对更具挑战性的语义细分任务进行彻底评估。将NAS应用于语义分割的主要挑战来自两个方面：（i）要处理的高分辨率图像； （ii）针对自动驾驶等应用的实时推理速度（即实时语义细分）的其他要求。为了应对此类挑战，我们在本文中提出了一种替代辅助的多目标方法。通过一系列自定义预测模型，我们的方法有效地将原始的NAS任务转换为普通的多目标优化问题。然后是用于填充选择的层次预筛选标准，我们的方法逐渐实现了一组有效的体系结构在细分精度和推理速度之间进行交易。对三个基准数据集的经验评估以及使用华为地图集200 dk的应用程序的实证评估表明，我们的方法可以识别架构明显优于人类专家手动设计和通过其他NAS方法自动设计的现有最先进的体系结构。

检测欺诈性交易是控制​​电子商务市场风险的重要组成部分。除了已经在生产中部署的基于规则和机器学习过滤器外，我们还希望使用图形神经网络（GNN）进行有效的实时推理，这对于在事务图中捕获多跃风风险传播非常有用。但是，在生产中实施GNN时出现了两个挑战。首先，在消息传递中不应考虑以预测过去中的动态图中的未来信息。其次，图形查询和GNN模型推断的延迟通常高达数百毫秒，这对于某些关键的在线服务来说是昂贵的。为了应对这些挑战，我们提出了一个批处理和实时的成立图拓扑（BRIGHT）框架，以进行端到端的GNN学习，以允许有效的在线实时推理。 Bright框架由图形转换模块（两阶段有向图）和相应的GNN体系结构（Lambda神经网络）组成。两阶段的指示图保证了通过邻居传递的信息仅来自历史支付交易。它分别由代表历史关系和实时链接的两个子图组成。 Lambda神经网络将推断分为两个阶段：实体嵌入的批次推断和交易预测的实时推断。我们的实验表明，在平均W.R.T.〜精确度中，BRIGHT优于基线模型> 2 \％。此外，BRIGHT在实时欺诈检测上在计算上是有效的。关于端到端性能（包括邻居查询和推理），BRIGHT可以将P99延迟降低> 75 \％。对于推理阶段，与传统GNN相比，我们的加速平均为7.8美元。

在本文中，我们提出了通过绘制物理信息监督残余学习（PHISRL）的方案来求解2D逆散射问题（ISP）来求解传统出版方法的计算过程（ TBIM）。 NeuralBim采用独立的卷积神经网络（CNNS）来学习两种不同候选解决方案的交替更新规则，其相应的残差。本文提出了两种不同的NeuralBim方案，包括监督和无监督的学习计划。利用由时刻（MOM）的方法生成的数据集，监督的NeuralBims培训具有总场的知识和对比度。无监督的NeuralBim是由ISP的管理方程式创建的物理嵌入式损失功能，这导致总场的要求和培训对比。代表性数值结果进一步验证了监督和无人育的NeuralBims的有效性和竞争力。

现有摘要系统主要生成纯粹依赖源文档内容的摘要。但是，即使对于人类，我们通常需要一些引用或示例，帮助我们充分了解源文档并以特定格式写入摘要。但是如何找到高质量的样式，并将它们纳入总结系统仍然挑战和探索。在本文中，我们提出了一种由致密的猎犬和摘要提升的新型检索增强的抽象概要框架。首先，检索几个密切相关的示例作为补充输入，以帮助生成模型更全面地了解文本。此外，检索的示例也可以在引导模型以捕获特定语料库的写入风格中起作用。我们在多个域和两个骨干型号的各种摘要数据集上验证我们的方法：BERT和BART。结果表明，与强大的预训练模型相比，我们的框架在胭脂-1分数中获得了1.38〜4.66的显着改善，并在账单上实现了新的最先进。人类评估表明我们的检索增强模型可以更好地捕获特定于域的书写风格。

在本文中，我们利用了以前的预训练模型（PTM）的优势，并提出了一种新型的中国预训练的不平衡变压器（CPT）。与以前的中国PTM不同，CPT旨在利用自然语言理解（NLU）和自然语言生成（NLG）之间的共同知识来促进表现。 CPT包括三个部分：共享编码器，一个理解解码器和一代解码器。具有共享编码器的两个特定解码器分别通过蒙版语言建模（MLM）进行了预训练，并分别将自动编码（DAE）任务进行了验证。借助部分共享的体系结构和多任务预培训，CPT可以（1）使用两个解码器学习NLU或NLG任务的特定知识，并且（2）对模型的潜力充分利用了微调。此外，不平衡的变压器节省了计算和存储成本，这使CPT竞争激烈，并极大地加速了文本生成的推断。对各种中国NLU和NLG任务的实验结果显示了CPT的有效性。

在线零售平台，积极检测交易风险至关重要，以提高客户体验，并尽量减少财务损失。在这项工作中，我们提出了一种可解释的欺诈行为预测框架，主要由探测器和解释器组成。 Xfraud探测器可以有效和有效地预测进货交易的合法性。具体地，它利用异构图形神经网络来从事务日志中的信息的非渗透键入实体中学习表达式表示。 Xfraud中的解释器可以从图表中生成有意义和人性化的解释，以便于业务部门中的进一步进程。在我们对具有高达11亿节点和37亿边缘的实际交易网络上的Xfraud实验中，XFraud能够在许多评估度量中倾销各种基线模型，同时在分布式设置中剩余可扩展。此外，我们表明，XFraud解释者可以通过定量和定性评估来显着帮助业务分析来产生合理的解释。